Statistical process control tools for monitoring clinical performance.

The quality assurance of medical practice in most countries This issue is of obvious concern for monitoring clinical practice, which requires early warning of poor peris effected through a mixture of informal assessment and peer review, and through more formal accreditation, creformance before too many patients are harmed. (3) Case-mix varies among practices, and adjustment for dentialing or delineation of privilege. The process of assessment and review is often subjective and without explicit case-mix is required to improve the accuracy of the signal. Such regression adjustment filters out the comreference to pre-determined standards of practice. It has been argued that comparative treatment outcome data on an ponent of variability in the outcome measurement induced by factors outside the process being monindividual doctor’s performance is required to make quality assurance credible [1]. Equally, many would add that objective itored, such as case-mix. This enhances the traceability of the source of the problem, and thus avoids efforts and quantitative methods to monitor the quality of a doctor’s performance based on treatment outcome data could be being wasted on investigating non-existent problems or, worse still, unfair accusation of poor performance more widely applied and would lend credence to the quality assurance process. toward practices with a large number of high-risk patients. Statistical process control (SPC) tools such as control charts have been widely used in the manufacturing industry for a long time. Also, the case for their application in health Experience in applying such SPC techniques as SPRT, as demonstrated in the paper by Spiegelhalter et al. in this issue care has long been made [2–4]. Resistance to their more widespread adoption in health care may stem from lack of [6], and CUSUM [7–14] have shown that these challenges could be met successfully. These experiences in applying evidence or lack of conviction regarding their power or utility. This issue of the International Journal for Quality in Health Care SPRT and CUSUM in clinical performance monitoring should also help to dispel the perhaps common misconception that contains yet another demonstration of the utility and power of such techniques. The application of risk-adjusted sequential such techniques could only detect extreme aberration in performance [15]. Indeed, one may argue that current pracprobability ratio tests (SPRT) retrospectively to two high profile examples, the Bristol Royal Infirmary pediatric cardiac tices that are based on subjective review or a simple monitoring scheme that does not exploit the power of modern surgery data and Harold Shipman’s data, demonstrate convincingly how in both cases the technique would have proSPC techniques have yet to demonstrate their capability in providing early warning of poor performance in clinical vided early warning of poor performance. This should argue cogently for the SPC case and add to its evidence base. practice. More statistical techniques are being added to our toolbox The application of SPC in the clinical context poses special challenges. The best-known control charts are those pioneered for use in clinical monitoring [16,17] and existing techniques are being refined [18]. The plethora of techniques may by Walter Shewart [5], such as his Xbar and R charts. Shewart charts, however, are designed to detect large but transient be confusing. While all these techniques are demonstrably effective in signaling poor performance—an obviously funshifts in the process mean, typically in large volume manufacturing processes. This limits their application in clinical damental requirement we must ask of them—they are not necessarily equally effective. There is one simple criterion for practice for several reasons: picking the optimal technique and its design for a given monitoring situation. Monitoring schemes such as SPRT, (1) The throughput of clinical practice is typically very slow; for example, a surgeon may perform no more CUSUM and others are designed to detect small changes in the treatment process. Unfortunately, this also means they than 1–5 procedures a day. It is both undesirable and inconvenient for a performance monitoring system to would allow an extremely tiny shift of no clinical importance to eventually produce a signal. In other words, a false alarm require sample sizes of greater than one to accumulate before analysis. may occur. Any monitoring system that is prone to frequent false alarm will be quickly discredited, and not be used. This (2) In clinical practice, even a small shift in process mean may be of concern; for example, it may indicate adverse is the inherent trade-off between sensitivity and false alarm in any monitoring system. We want the system to be sensitive deterioration in mortality rate or complication rate.